Bias-Variance Tradeoff
Formula
$$ \begin{aligned} \mathbb{E}\left[ (\hat{f} - y)^2\right] = \left(\text{Bias}\left[ \hat{f}\right]\right)^2 + \text{Var}(\hat{f}) + \sigma^2, \begin{cases} \text{Bias}\left[ \hat{f}\right] = \mathbb{E}\left[ \hat{f} - y\right] \\ \text{Var}\left[ \hat{f}\right] = \mathbb{E}\left[ \hat{f}^2\right] - \left(\mathbb{E}\left[ \hat{f} \right]\right)^2 \end{cases} \end{aligned} $$where $\hat{f}$ is the estimation of $y = f + \epsilon, \mathbb{E}\left[ \epsilon\right] = 0$.
Proof
$$ \begin{aligned} \mathbb{E}\left[ (\hat{f} - y)^2\right] &= \mathbb{E}\left[ y^2 + \hat{f}^2 - 2y\hat{f} \right] \\ &= \mathbb{E}\left[ y^2 \right] + \mathbb{E}\left[ \hat{f}^2 \right] - \mathbb{E}\left[ 2y\hat{f} \right] \\ &= \text{Var}(y) + \left(\mathbb{E}\left[ y \right]\right)^2 + \text{Var}(\hat{f}) + \left(\mathbb{E}\left[ \hat{f} \right]\right)^2 - \mathbb{E}\left[ 2f\hat{f} \right] \\ &= \text{Var}(y) + \text{Var}(\hat{f}) + \left[ \left(\mathbb{E}\left[ y \right]\right)^2 - \mathbb{E}\left[ 2y\hat{f} \right] + \left(\mathbb{E}\left[ \hat{f} \right]\right)^2 \right] \\ &= \text{Var}(y) + \text{Var}(\hat{f}) + \left[ y^2 - 2y\mathbb{E}\left[ \hat{f} \right] + \left(\mathbb{E}\left[ \hat{f} \right]\right)^2 \right] \\ &= \text{Var}(y) + \text{Var}(\hat{f}) + \left( f - \mathbb{E}\left[ \hat{f} \right] \right)^2 \\ &= \sigma^2 + \text{Var}(\hat{f}) + \left( y - \mathbb{E}\left[ \hat{f} \right] \right)^2 \\ &= \sigma^2 + \left(\text{Variance}\right) + \left( \text{Bias} \right)^2 \\ \end{aligned} $$Note that $y, \hat{f}$ are $f(x) + \epsilon, \hat{f}(x)$, which is for a single $x$. Notice that $f$ is deterministic, thus $\mathbb{E}\left[y\right] = \mathbb{E}\left[ f\right] = f$.